Editing system with router for connection to HDTV circuitry

ABSTRACT

A non-linear editor is connected to video processing equipment through a serial digital video interface to edit high definition (HD) television video data. The non-linear editor includes a randomly accessible, computer-readable and re-writeable storage medium that stores a plurality of sequences of HD digital images representing a frame or field of HD motion video data. The non-linear editor provides a configuration control signal to identify processing to be performed on the HD video data and defines a video program to be rendered using the stored HD digital images. An input serial digital interface and an output serial digital interface in the non-linear editor provide the HD video data to be edited. A multiformat video router controls the HD video data sent between the non-linear editor and the video processing equipment. The router is video interconnected to the video processing equipment and to the serial digital interfaces of the non-linear editor.

This application is a continuation of application Ser. No. 09/800,867filed Mar. 7, 2001, now abandoned.

FIELD OF THE INVENTION

The present invention is related to a system for editing high definitiontelevision (HDTV) resolution video data.

BACKGROUND OF THE INVENTION

Separate editing systems exist for editing standard definitiontelevision (SDTV) resolution video data and for editing HDTV-resolutionvideo data. Currently, there is a need for a HDTV editor formanipulating digital high definition (HD) video data which can beconfigured from a set of editing devices such as a non-linear editorvideo interconnected to video processing equipment.

SUMMARY OF THE INVENTION

A non-linear editor is connected to video processing equipment through aserial digital video interface (SDI) to edit high definition televisionvideo data.

Accordingly, in one aspect a system edits HDTV-resolution video data. Inthe system, a non-linear editor includes a random access,computer-readable and re-writeable storage medium that stores aplurality of sequences of high definition (HD) digital still images inmedia files. Each image may represent a single frame, i.e., two fields,or a single field of HD video data. The non-linear editor provides aconfiguration control signal to define a video program to be renderedusing the stored HD digital still images. The non-linear editor includesan input serial digital interface and an output serial digital interfaceto provide the HD video data to be edited. In the system, a multiformatvideo router directs the HD video data between the non-linear editor andvideo processing equipment. The router is connected by a videointerconnect to the input serial interface and the output serialinterface of the non-linear editor and the router receives theconfiguration control signal from the non-linear editor. The router isconnected by a video interconnect to provide video data to an input ofthe video processing equipment, and is connected by a video interconnectto receive a video data output of the video processing equipment. In thesystem, video processing equipment has an input for receiving HD videodata from the multiformat router, and an output for sending HD videodata to the multiformat router. The video processing equipment also hasan input for receiving the configuration control signal from thenon-linear editor.

Another aspect is a non-linear editor which may include a plurality ofHD serial digital interfaces to convert a video stream to bus data.

Another aspect is-video processing equipment including at least one HDcodecs for formatting the HD video data and a video effects generatorfor editing the HD video data.

Another aspect is a multiformat video router which includes a standarddefinition router and an HD router.

Another aspect is a method for storing edited HDTV-resolution videodata. Edited HD video data is received from a video effects generator.The edited HD video data is compressed into an HD video data file whichincludes HD video data information. The compressed HD video data file issent through a serial digital interface to a non-linear storage system.

Another aspect is a method for playing back HD video data. HD video datais retrieved from storage and transferred through a serial digitalinterface. The retrieved HD video data is decompressed and sent to avideo effects generator for processing. The processed HD video data isprovided as an output.

Another aspect is buffering the HD video data at an HD serial interfaceand providing the buffered HD video data to the video effects generator.

Another aspect is capturing the output of the video effects generator atan HD serial digital interface.

Another aspect is a method for editing HDTV-resolution video data. Aplurality of sequences of HD digital still images is stored and a videoprogram to be rendered using selected stored HD digital still images isdefined. Devices including video processing equipment are configured forprocessing the selected HD still images. The selected HD still imagesare transferred over a video interconnection to the video processingequipment and the processed HD still images are rendered.

BRIEF DESCRIPTION OF THE DRAWING

In the drawing,

FIG. 1 a is a block diagram of a system in one embodiment;

FIG. 1 b illustrates a table for tracking equivalency of media datafiles;

FIG. 2 is a more detailed block diagram of the system in FIG. 1;

FIG. 3 is a flowchart describing how video effects are played back inthe system of FIG. 2; and

FIG. 4 is a flowchart describing how video with video effects is storedin the system of FIG. 2.

DETAILED DESCRIPTION

FIG. 1 a is a block diagram of an example system for editing highdefinition (HD) video data. HD video data may include any data at aresolution higher than standard definition (SD) video data, for example,data with a resolution greater than 525 scan lines and/or at more than30 frames/sec. The HD data may be in 8 or 10-bit components. The systemincludes video processing equipment 110 which processes HD video data,and a router 120 which transfers HD video data to the video processingequipment 110 from the editing system 130.

The video processing equipment 110 may be, for example, one or morecoder/decoder processors (codecs), a video effects generator or displayor capture device. Video processing equipment 110 may capture highdefinition (HD) video data for processing at input 118. Video to beplayed back may be output from video processing equipment 111 at output114.

Video router 120 may be a multiformat router (e.g., a router capable ofdirecting both standard (SD) and HD video data) which is an interfacefor the HD video data which travels between the video processingequipment 110 and the editing system 130 through inputs and outputs 114,118, 124 and 128. Router 120 may also be two separate routers—one for HDvideo data and one for standard definition (SD) video data. Router 120may be a cross-point switch such as the HDS-V3232 by Sony Corporation.Router 120 is configurable by the editing system 130 based on theediting to be performed.

A configuration control signal 134 may be sent by editing system 130 torouter 120 and video processing equipment 110 to configure those devicesaccording to the type of editing to be performed and the amount of HDvideo data to be edited.

The editing system 130 is a non-linear editor including a random-access,computer-readable and re-writeable storage medium that stores a sequenceof digital still images. Each still image may represent a single frame,i.e., two fields, or a single field of motion video data. The editingsystem 130 may allow any particular image in the sequence of stillimages to be randomly accessed for playback. The images may includeuncompressed video data, however, since digital data representing motionvideo may consume large amounts of computer memory, the digital datatypically is compressed to reduce storage requirements.

Various types of compression may be used. Some kinds of compression mayoperate on a stream of data regardless of how the data may be divided todefine an image. One kind of compression is called “intraframe”compression which involves compressing the data representing each stillimage independently of other still images. Commonly-used motion videocompression schemes using intraframe compression include “motion-JPEG”and “I-frame only” MPEG. Intraframe compression allows purely non-linearaccess to any image in the sequence.

More compression can obtained for motion video sequences by using whatis commonly called “interframe” compression. Interframe compressioninvolves predicting one image using another. This kind of compressionoften is used in combination with intraframe compression. Severalstandards use interframe compression techniques, such as MPEG-1 (ISO/IEC11172-1 through 5), MPEG-2(ISO/IEC 13818-1 through 9) and H.261, anInternational Telecommunications Union (ITU) standard. MPEG-2, forexample, compresses some images using intraframe compression (calledI-frames or key frames), and other images using interframe compressiontechniques for example by computing predictive errors between images.The predictive errors may be computed for forward prediction (calledP-frames) or bidirectional prediction (called B-frames). MPEG-2 isdesigned to provide broadcast quality full motion video. Interframecompression does not allow purely non-linear access to every image inthe sequence, because an image may depend on either previous orfollowing images in the sequence. The invention is not limited to aparticular kind of compression and does not require compression.

There are several kinds of systems that may be used to author, processand display multimedia data. These systems may be used to modify thedata, define different combinations of data, create new data and displaydata to a user. A variety of techniques are known in the art forimplementing these kinds of systems.

Multimedia authoring, processing and playback systems typically have adata structure which represents the multimedia composition. The datastructure ultimately refers to clips of source material, such asdigitized video or audio, using an identifier of the source material,such as a unique identifier or a file name, and possibly a temporalrange within the source material defining the clip. The identifier maybe of a type that may be used with a list of equivalent data files toidentify a file name for the source material. An index may be used totranslate the temporal range in the source into a range of bytes withina corresponding file. This range of bytes may be used with the segmenttable for the file to identify segments of data that are needed and thestorage units from which the data is retrieved.

FIG. 1 b shows an example list structure that may be used to representpart of a multimedia composition that may be created by editing system130. In an example shown in FIG. 1 b, there are several clips 560, eachof which includes a reference to a source identifier, indicated at 562,and a range within the source, as indicated at 564. Generally, there maybe such a list for each track of media in a temporal composition. Thereare a variety of data structures which may be used to represent acomposition. In addition to a list structure, a more complex structureis shown in PCT Published Application WO93/21636 published on Oct. 28,1993. Other example representations of multimedia compositions includethose defined by Open Media Framework Interchange Specification fromAvid Technology, Inc., Advanced Authoring Format from the MultimediaTask Force, QuickTime from Apple Computer, DirectShow from Microsoft,and Bento also from Apple Computer, and as shown in PCT PublicationWO96/26600.

The data structure described above and used to represent multimediaprograms may use multiple types of data that are synchronized anddisplayed. The most common example is a television program or filmproduction which includes motion video (often two or more streams ortracks) with associated audio (often four or more streams or tracks).

Because the video and audio data may be stored in different data filesand may be combined arbitrarily, better performance may be obtained ifrequests for data for these different data files are managedefficiently. For example, an application may identify a stream for whichdata can be read, and then may determine an amount of data which shouldbe read, if any. A process for performing this kind of management ofread operations is shown in U.S. Pat. No. 5,045,940. In general, theapplication determines which stream has the least amount of dataavailable for display. If there is a sufficient amount of memory data tobe played back for that stream to efficiently read an amount of data,then that data is read from the file. When it is determined that datafor a stream should be requested, each segment of the data is requestedfrom a storage unit selected from those on which the segment is stored.In order to identify which files to request from the storage unit, theediting system may convert a data structure representing a composition,such as shown in FIG. 1 b, into file names and ranges within thosefiles.

Editing system 130 may use various audio and video media files stored ona storage system to create a composition. Editing system 130 may becapable of handling one or more tracks of audio/video information, andmay be capable of performing editing functions such as dissolves, wipes,flips, flops, and other functions known in the art of video production.Media files are typically created by a digitizing system (not shown)that receives one or more audio/video inputs from a media player (notshown). These media files may also be digitized directly by a digitalrecorder (not shown). Editing system 130 may also use interactiveelements in creating a composition.

A commercial editing system 130 may be used, such as the Media Composervideo production system or NewsCutter news editing system available fromAvid Technology, Inc. (NewsCutter is a registered trademark of AvidTechnologies, Inc.). Also, a commercial playback system suitable forimplementing the present invention may be used that implements the MediaEngine video playback system available from Avid Technology, Inc. thatis incorporated in the Avid AirPlay MP playback server system (AirPlayis a registered trademark of Avid Technology, Inc.). A commercialstorage system (not shown) suitable for storing composition filesincludes the MediaShare external storage device (MediaShare is atrademark of Avid Technology, Inc.). Other commercial systems may beused.

FIG. 2 is a more detailed block diagram of a system for editing highdefinition video data such as the one shown in FIG. 1 a. A non-lineareditor is shown as computer 210 and non-linear storage system 205. Innon-linear systems, the need for multiple copies of video sources toproduce arbitrary sequences of segments has been avoided by therandom-access nature of the media. Arbitrary sequences of segments frommultiple data files are provided by pipelining and buffering non-linearaccesses to the motion video data. Storage system 205 stores HD videodata in compressed format as media files, although the HD video data mayalso be in uncompressed format. Another example of an editing system maybe found in U.S. Patent Application entitled “HDTV EDITING ANDPREVISUALIZATION USING SDTV DEVICES” by Craig R. Frink et al. filed Apr.3, 1998.

Computer 110 includes a serial digital interface (SDI) and a highdefinition serial digital interface (HD-SDI). The SDI and HD-SDIinterfaces provide video interconnections to router 120. The SDI cardsmay be, for example, the Dynamo VideoPump card by Viewgraphics, Inc, oran SDI card by Gennum. From the point of view of the non-linear editor,the SDI is a video input and output device. The SDI cards can transfermultiple streams of HD video data concurrently and in real-time to thestorage system 205.

The HD-SDI cards may be any interface card which can capture an HD videostream at a rate in the range of 54 million to (480 Progressive) 148.5million components/second (1080 interlaced), (e.g., 8 or 10 bits ) andto convert the HD video stream to peripheral connection interface (PCI)type bus data. A 64 bit/33 MHZ PCI bus or a 32 bit/66 MHZ PCI may becapable of transferring HD data in real time thereby minimizing thebuffer size requirements. Each of the HD-SDI cards has a buffer capableof capturing a number of high definition (HD) video frames which maylater be transferred for processing or storage. The cards may beexpandable to accommodate additional codec or video effects generatorequipment which include more inputs and outputs of HD video data. One ofordinary skill in the art may develop an HD-SDI card based on knownproducts by Sony and Panasonic which include both SDI and HD-SDIinterfaces, from known HD products, or from technology used for SDIcards by Viewgraphics and Gennum. The SDI and HD-SDI cards provide avideo interconnection between computer 210 and routers 215 and 220. Thevideo interconnection between the SDI cards and router 215 allowscompressed HD video data representing an image to be edited to becarried in packet form between the non-linear editor 210 and HD codecs230 and 240. The video data transferred by the SDI is defined usingmarkers signifying the Start of Active Video (SAV) and End of ActiveVideo (EAV) to delineate a field of HD video data. The videointerconnection between the HD-SDI cards and router 220 allows acontinuous, uncompressed HD video data stream to be carried between theHD-SDI cards and router 220.

A computer interconnection between the interface cards and the routersmay also be used instead of a video interconnection. The computerinterconnection assigns an address for each device in the system anduses control information to identify a start of a frame of HD video dataand a number of lines which is being sent to coordinate the transfer ofthe HD video data between the devices. When using a computerinterconnection, the non-linear editor is responsible for identifyingeach device and its address in the system. However, when the videointerconnection is used, the non-linear editor is responsible forproviding an output HD video data stream. Therefore, the devices whichreceive or send the HD video data stream, as well as other devices inthe system, are transparent to the non-linear editor.

Router 215 transfers compressed high definition video data betweencomputer 210 and video processing equipment which may include videoeffects generator 245 and high definition codec 230 or high definitioncodec 240. Router 215 may be connected to an input/output port 225 forreceiving and transferring compressed HD video data. Router 215 may alsobe connected to an external videotape recorder (VTR) 235, such as a D-5VTR from Panasonic, to store HD video data. Router 215 may be used totransfer compressed HD data and may be, for example, a Society of MotionPicture and Television Engineers (SMPTE) standard SMPTE-259 router, suchas a DVS-V 1616 by Sony or a VIA 32 series router such as VIA 16×16s byLeitch. The input and the output side of router 215 may be configurable,for example, by increments of four channels.

Router 220 directs uncompressed high definition video data betweencomputer 210 and video effects generator 245. Router 220 may be aSMPTE-292 router for transferring uncompressed HD video data. Router 220is also connected to HD video effects generator 245 which operates onreal-time video streams through input/output port 270 used fortransferring digital high definition video signals to or from externalequipment. Video effects generator 245 may be, for example, a productdeveloped by Sony, Grass Valley or Abekas. Router 220 is connected tohigh definition digital to analog (D/A) converter 250 which provides anoutput to high definition video monitor 275 or to analog high definitionoutput 260. HD video monitor may be for example, Sony's HDM2830 orPanasonic's AT-H3215W plus a digital to analog convertor such asPanasonic Model AJ-HDA500. Router 220 includes an input 244 from highdefinition analog to digital (A/D) converter 255 which receives ananalog high definition input 265.

Both routers 215 and 220 are configurable by the non-linear editor whichspecifies the type of editing to be performed and the amount of HD datato be edited. The routers 215 and 220 transfer the HD video data basedon the configuration control signal. The editing process switchesbetween playback and storage of the edited HD video data (the processesare described below in connection with FIGS. 3 and 4) and the routerschange their routing configuration based on a time code or a number offrames to be played back or stored as indicated by the non-linear editorduring the initial configuration.

HD codecs 230 and 240 may be, for example, the Digital HD VTR Processorby Panasonic, part number AJ-HDP500P. The codecs compress and decompressthe HD video data. The operation of FIG. 2 will now be described inconnection with FIGS. 3 and 4.

FIG. 3 is a flowchart of the process of editing HDTV video data andplaying back the edited HD video data. By defining a sequence of clipsof video data, high definition video data is retrieved by non-lineareditor 210 from a storage system 205, step 305. Tie HD video data may beretrieved as a single field or frame of HD video data or as a linear ornon-linear sequence of video clips. The storage system 205 may benon-linear and may allow random non-linear access of HD video data.

The retrieved HD video data is transferred by the SDI card to router 215and is sent to codec 230 or codec 240, step 310. In a single streamsystem only codec 230 is used. Codec 230 receives the retrieved HD videodata, which is in the format of a compressed data file representing theHD video image, and decompresses the HD data file to video data format.

In step 315, if the effect to be applied is determined to be an A/Beffect (i.e., a wipe, fade, etc.) then the uncompressed video is sentthrough router 220 to video effects generator 245, step 320. If theeffect is not an A/B effect, and is for example, an A/B/C effect (i.e.,a ripple, page curl, etc.), the uncompressed video is sent to an HD-SDIinterface card, step 325 where it is buffered while the system waits foranother clip of HD video data to be used in creating the effects. Afterthe streams of HD video data for the A/B/C effect is buffered, the HDvideo data is sent to the video effects generator 245 in step 330 forprocessing.

The HD video data output of the video effects generator 245 whichincludes the added effects is captured by the HD-SDI card in thenon-linear editor 210. The non-linear editor 210 may also be used toedit the HD video data. The HD video data output of the video effectsgenerator 245 may be sent to HD video monitor 275 or it may betransferred as an analog 260 or digital 270 HD output. The process ofstoring the generated HD video is described below.

FIG. 4 is a flowchart of the process of storing an output of the videoeffects generator 245 to disk. In step 405, the rendered video from thevideo effects generator 245 is sent to the HD-SDI card where it isbuffered.

If there are more effects to be added in step 415, then additional videois sent to the video effects generator 245 with the rendered video toallow more complex editing, in step 410. After all of the desiredeffects have been added to the HD video data in step 415, the HD videodata is sent through a codec, such as codec 230 in step 420 where the HDvideo data with the effect is transformed to a compressed format. Instep 421, the compressed HD video data is next transferred to an SDIcard in the non-linear editor 210. The edited HD video data istransferred to storage system 205 in step 422. A composition which hasbeen edited and stored according to the process in FIG. 4 can be playedback using the process in FIG. 3, such that the data file including theedits is played back, rather than separate sources of HD video data.

Having now described a few embodiments, it should be apparent to thoseskilled in the art that the foregoing is merely illustrative and notlimiting, having been presented by way of example only. Numerousmodifications and other embodiments are within the scope of one ofordinary skill in the art and are contemplated as falling within thescope of the invention.

1. A system for editing high definition television resolution video datacomprising: a non-linear editor including a random-access,computer-readable and re-writeable storage medium that stores aplurality of sequences of digital still images representing highdefinition video data in media files, wherein the non-linear editorprovides a configuration control signal that defines a video program tobe edited using the stored high definition digital still images, andwherein the non-linear editor includes a serial digital interface forinput and output of the high definition video data to be edited; arouter for directing the high definition video data between thenon-linear editor and a video processing equipment, wherein the routeris connected to the serial digital interface, wherein the routerreceives the configuration control signal from the non-linear editor toinitialize the configuration of the router, wherein the router isconnected to provide video data to the video processing equipment and toreceive video data from of the video processing equipment; and whereinthe video processing equipment has an input for receiving highdefinition video data to be edited from the router and an output forsending edited high definition video data to the router, and has aninput for receiving the configuration control signal from the non-lineareditor to determine processing to be performed on the received highdefinition video data.
 2. The system of claim 1, wherein the videoprocessing equipment includes a video effects generator for editing thehigh definition video data.
 3. The system of claim 1, wherein the outputof the video processing equipment is transferred to a high definitionvideo monitor.